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Abstract Observational astronomy has shown significant growth over the last decade and 
has made important contributions to cosmology. A major paradigm shift in cosmology 
was brought about by observations of Type la supernovae. The notion that the universe is 
accelerating has led to several theoretical challenges. Unfortunately, although high qual- 
ity supernovae data-sets are being produced, their statistical analysis leaves much to be 
desired. Instead of using the data to directly test the model, several studies seem to con- 
centrate on assuming the model to be correct and limiting themselves to estimating model 
parameters and internal errors. As shown here, the important purpose of testing a cosmo- 
logical theory is thereby vitiated. 
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Observations play a crucial role in all branches of science. Moreover, they are more important in cos- 
mology where, on one hand, the events are non-repeatable and, on the other hand, the theoretical side 
is more speculative than laboratory physics, requiring guidelines from the observations by confronting 
them. The power of observations in cosmology is clear from the observations of supernovae (SNe) of 
type la, which dramatically changed, about a decade ago, the then standard picture of cosmology - of 
an expanding universe evolving under the rules of general relativity such that the expansion rate should 
slow down as cosmic time unfolds. Amazingly, the first generation of data showed that the rate of expan- 
sion of the universe is speeding up! This gave rise to a plethora of models to explain the agent driving 
the acceleration, as well as some possible modifications of general relativity. Theorists may debate the 
relative merits of various cosmic-acceleration theories: cosmological constant, dark energy, alternative 
gravity, anthropic arguments, etc., but it is ultimately up to the observations to decide which theory is 
correct. 

However, we notice a recent unfortunate trend in the analysis of SNe la data, which departs from 
an objective assessment of a theory by observations. Everybody would agree that the first step in fitting 
the observational data to a theory is to check whether the theory is consistent with the data for viable 
values of its free parameters (if any). In order to do this, there are two standard ways: (i) the Bayesian 
approach, which gives a relative rather than an absolute measure of how good a theory is, and hence is 
more appropriate for comparison between competing theories; (ii) the maximum likelihood approach, 
which is more commonly used for hypothesis testing. Under this approach, one minimises \ 2 which 
measures the deviations of the theoretical predictions from the observations. In the present case of the 
observations of magnitude (m) and redshift (z) of SNe la, the \ 2 is given by the value 
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where mt(zj; parameters) is the theoretical value of the magnitude at redshift Zj of the jth supernova 
predicted by a model which is given in terms of its parameters. The observed magnitude of the j-th SN 
is m ,j- The variance of n represents the combined uncertainty in the observed magnitude of the jth 
supernova arising from the uncertainties in the different variables, for example, lensing, dust extinction, 
peculiar velocity of the host galaxy, etc. By Taylor expanding m around its mean value and by recalling 
that the variance of (m) = = (m 2 ) — (m) 2 , one can write the combined uncertainty of m in terms 
of the uncertainties in its variables, say, xf. 
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where cov(xj, Xk) is the covariance between the variables Xi and Xk, which vanishes for any uncorre- 
cted variables. It is obvious from equation (Q~|i that if the model satisfactorily represents the data, the 
difference between the predicted magnitude and the observed one at each data point should be roughly 
the same size as the measurement uncertainties and each data point would contribute roughly one to x 2 , 
giving the sum that roughly equals the number of data points TV (more correctly TV— number of fitted 
parameters = number of degrees of freedom 'DoF'). If \ 2 is much larger, the fit is considered bad. A 
more quantitative assessment of the goodness-of-fit is given in terms of the x 2 -probability. If the fitted 
model provides a typical value of \ 2 as i at n DoF, this probability is given by 
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P (^ n ) = W^I e-V/^du. (3) 
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Thus, P(x, n) gives the probability that a model which does fit the data at n DoF, would give a value 
of x 2 as large as x or larger. Qualitatively, P represents the probability of finding a worse fit to the 
data. If P is very small, the fit is not acceptable. For example, if we get a \ 2 = 20 at 4 DoF for 
some model, then the hypothesis that the model describes the data satisfactorily is unlikely, as the 
probability P(20, 4) = 0.0005 is very small. However, the \ 2 -probability P holds strictly only when 
the measurement errors are normally distributed. In reality, they are not. In most cases, the effect of 
non-Gaussian errors is to create an abundance of outlier points, which decrease the probability P. It is 
due to this reason that usually models with P > 0.001 are considered acceptable. One should proceed 
to estimate the parameters of the model only after examining if the model has a credible goodness-of-fit, 
in the absence of which, the estimated parameters of the model (and their estimated uncertainties) have 
no meaning at all. 

Nevertheless, we noticed (see also Vishwakarma 2007) that recently an approach has been followed 
to analyze the SNe la data which does not respect the standard procedure described above. Initiated by 
Astier et al. (2006), this approach simply assumes (rather than examines) that the standard cosmology 
(23% of dark matter and 72% of dark energy) is consistent with the SNe la observations and limits itself 
to parameter estimation. Under this new approach, \ 2 is calculated from 
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where er int is the (unknown) intrinsic dispersion of the SN absolute magnitude (sometimes termed cr sys , 
unknown systematic uncertainties) which is used as an adjustable free parameter in order to obtain 
X 2 /DoF = 1 . Clearly this new approach does not test the model under consideration. Rather, it assumes 
that the model is correct and goes on to find the best value for intrinsic dispersion. In principle, any model 
applied to the data in this way will determine a- m t and the purpose of testing the model or estimating its 
parameters is lost. 

One may consider x 2 /DoF = 1 to estimate a- m t from the nearby SNe alone as cr int estimated from the 
nearby SNe alone and that estimated from the whole sample, do not differ significantly. For example, 
o'int = 0.15 ± 0.02 estimated from the fit of the nearby sample (z < 0.1) of Astier et al. (2006) 
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appears to be statistically consistent with (Tint = 0.13 ± 0.02 estimated from their whole sample and 
Cint = 0.12 ± 0.02 estimated from their distant SNe alone. However, one can always argue why a 
cosmological theory should be assumed to be correct (i.e., x 2 /DoF =1) even for the nearby SNe, even 
though all the Robertson-Walker models reduce to the m% = 5 log z + constant for low z. In addition, 
a more reasonable way to introduce <7i nt in the theory, is to use independent measurement uncertainties 
Cint j from the individual SNe and use the correct expression for cr^ given by equation (O where <Ti„ t 
enters through the term [dm / dxi nt ) 2 of nt . 

Of course, one can estimate cr- mt (if one is just interested in that) from all (high- as well as low- 
redshift) SNe data by assuming that a particular theory (here the standard cosmology), already tested, 
must be consistent with the data (i.e., x 2 /DoF = 1). This is fine if the theory is well established and is 
already tested by other independent ways. Then we are not interested in testing the already established 
theory, but rather we want to estimate, from©, some parameters of the data (here erj nt ) which we 
could not decipher from the observations. This procedure is followed in many branches of physics. But 
what are the other observations which predict dark energy independently and conclusively? To date, 
it is only the SNe la observations (if taken independently, i.e., not combined with other observations). 
The only other precise observations are the anisotropy measurements of the CMB made by the WMAP 
project. However, taken at their face values, the only relevant prediction of the WMAP observations is 
of a flat geometry, and the decelerating models (like the Einstein-de Sitter model: VL m = 1, A = 0) 
also successfully explain them (see for example, Blanchard 2003; Blanchard et al. 2005; Vishwakarma 
2003). Observations on gravitational lensing, quasars, galaxy clusters, gamma ray bursts, etc., are not 
precise enough. Age considerations depend heavily on the observations of Ho and Q m which have 
wide-ranging degeneracy. Moreover, the best estimates of these parameters also include those which are 
estimated from the SNe la observations themselves. In fact, a conclusive prediction of dark energy only 
comes from the SNe la case. To make the situation worse, there have been claims (Wei 2010) of some 
conflicts between recent SNe la data sets Union (Kowalski et al. 2008) and Constitution (Hicken et al. 
2009); these disagree not only with CMB and BAO (baryon acoustic oscillation) studies, but also with 
other SNe la data sets. 

So, if we do not have conclusive independent evidence for dark energy, what is the meaning of 
(Tint (and other parameters) estimated from this model, following this new approach to data analysis? 
The current concordance model in cosmology (also known as ACDM cosmology) has 18 parameters 
(one can extract these parameters from Hinshaw et al. 2009 and Kowalski et al. 2008), 17 of which 
are independent. Thirteen of these parameters are well fitted to the observational data; the other four 
remain floating. This situation is very far from healthy, not only because of very few observational tests 
which support the standard cosmology, that it calls for more observational tests, but also because of the 
controversial theoretical aspects. The well known fine tuning and coincidence problems related to dark 
energy, as well as its extremely speculative character (particle physics even fails to identify a theory 
for dark energy), have invoked possible modifications of general relativity. Additionally on the observa- 
tional side, there remains a lack of understanding of some issues related to SNe la observations causing 
a number of systematic uncertainties (which are likely to depend on redshift) that could affect the use 
of SNe la as standard candles in such cosmological probes. For example, one can mention the evolution 
of luminosity in SNe la (Dominguez et al. 1998; Hoflich et al. 1998; Drell et al. 2000; Timmes, Brown 
& Truran 2003) and extinction of SNe light by dust which is still a poorly understood phenomenon 
(Holwerda 2008; Albrecht et al. 2006; Conley et al. 2007). Finally, it may be noted that different meth- 
ods for analyzing SNe la data, in order to estimate magnitudes, do not seem entirely consistent with each 
other, causing systematic error concerns. For example, two different light-curve fitters, the multi-color 
light-curve shape (MLCS) method and the spectral-adaptive light-curve template (SALT) method, give 
significantly different estimates of magnitudes of the individual SNe; besides, the magnitudes estimated 
by the SALT fitter acquire a degree of cosmology-dependence (Frieman 2008). This situation warrants 
more rigorous tests of the theories as well as the observations. Certainly it suggests caution in regard to 
accepting the standard model as proven. 

Initiated by the SuperNova Legacy Survey "SNLS" (Astier et al. 2006) and followed by Unionl 
(Kowalski et al. 2008), Union2 (Amanullah et al. 2010), Sloan Digital Sky Survey-II "SDSS-II" (Kessler 
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et al. 2009) and SNLS 3-year (Guy et al. 2010) data sets, this flawed approach has already acquired the 
status of 'standard practice' in Lampeitl et al. (2010). The harmful side effect of the current approach is 
that even if the authors do not follow this approach, usually they do not care to check if their model fits 
the data or not, and limit themselves to estimating the parameters of the model either by only using the 
SNe la data or by combining the SNe la data with other observations. This is clear from the results of 
the Constitution data (Hicken et al. 2009), the largest sample so far consisting of 397 SNe la: although 
the authors do not follow the new approach, they do not seem to notice that the theory does not fit the 
data well! One can calculate from their Table 1 that the best-fitting ACDM model, with fi m =l — ft\= 
0.29, gives x 2 /DoF = 465.5/395 with a meager probability P = 0.83%; and so the estimated model 
can be ruled out at more than a 99% confidence level! Other cosmologies also have a similar fit. 

Finally, we emphasize that our arguments here offer a criticism of the statistical technique used for 
analyzing the data rather than a criticism of the standard model itself. 
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